VIA V3 - The Custom UI Update #18222

wilba · 2022-08-30T19:09:49Z

VIA V3 - The Custom UI Update

This PR contains the changes to VIA code in QMK to support VIA V3 protocol definitions.

Custom UI, i.e. defining UI controls in VIA to control firmware
Full support for RGB Matrix
Delays in macros
Lots of refactoring/deleting redundant code in keyboards

In VIA Version 1,the UI and the keyboard definitions were entirely self-contained (i.e. hard-coded). Adding new keyboards requires changing code and rebuilding/releasing the app.

VIA Version 2 moved the keyboard definitions into externally defined files, which are stored in a GitHub repository. The keyboard definitions are served to the VIA client at run-time. This allowed anyone to add new keyboards to VIA without requiring changing the source and rebuilding/releasing the app.

VIA Version 2 keyboard definitions include layout options and control of lighting, but definitions were restricted to the existing UI elements.

VIA Version 3 is a refactoring of how the UI works in VIA, to allow fully customized UI within VIA to control firmware parameters like lighting, but also any custom feature implemented in the firmware, either in QMK Core or at the keyboard level. It works by defining what UI elements VIA should show, and binding those UI elements to IDs, which VIA will use in communication with the firmware.

The full documentation of the changes in VIA V3 can be read here:

https://www.caniusevia.com/docs/v3_changes

https://www.caniusevia.com/docs/custom_ui

The VIA client available at https://usevia.app currently works with V2 and V3 definitions. The V2 definitions still exist in the-via/keyboards/src. These were refactored into V3 definitions in the-via/keyboards/v3. Thus building firmware from this branch, which would require a V3 definition, should work in the VIA app.

Note: while the amount of code changes here seems large, it pales in comparison to the awesome amount of coding that @olivia has done to make VIA what it is today, especially the custom UI feature, the web app, and all the backend optimization that allows VIA to handle 4,000+ sessions a day. Special thanks also to @mattsacks and @ValdisThomann for development, and @yiancar and @Xelus22 for maintenance work. <2

Types of Changes

Issues Fixed or Closed by This PR

Checklist

My code follows the code style of this project: C, Python
I have read the PR Checklist document and have made the appropriate changes.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have read the CONTRIBUTING document.
I have added tests to cover my changes.
I have tested the changes and verified that they work and don't break anything (as well as I can manage).

drashna · 2022-08-30T19:16:52Z

Retargeted to develop, per contribution guidelines.

Also, lib/ugfx needs to be removed, and lint wants a format pass of the code.

Also, ideally, with this change, the keycode definitions need to be updated.

adophoxia · 2022-10-07T00:08:48Z

Is there anything else needed besides resolving merge conflicts and having it successfully run the QMK CI to have this be on the way to being merged?

drashna · 2022-10-07T03:01:20Z

Well, ideally, a bunch of the bodges to ensure that keycodes match with VIA should be removed (such as the mousekeys, and midi), which also requires the via_ensure_keycode.h file to be updated as well.

There has also been added a good number of keycodes added that should be supported tooand added to the "ensure" file, too

adophoxia · 2022-10-07T03:02:26Z

Ah ok. That's actually good to know.

drashna · 2022-10-07T06:36:44Z

Also, the macro section should be merged. via's macros and user keycodes could be safely moved to this block:

qmk_firmware/quantum/quantum_keycodes.h

Lines 562 to 594 in a8d6b37

    
           // Dedicated macro keys for Configurator and VIA 
        
           MACRO_0, 
        
           MACRO_1, 
        
           MACRO_2, 
        
           MACRO_3, 
        
           MACRO_4, 
        
           MACRO_5, 
        
           MACRO_6, 
        
           MACRO_7, 
        
           MACRO_8, 
        
           MACRO_9, 
        
           MACRO_10, 
        
           MACRO_11, 
        
           MACRO_12, 
        
           MACRO_13, 
        
           MACRO_14, 
        
           MACRO_15, 
        
           MACRO_16, 
        
           MACRO_17, 
        
           MACRO_18, 
        
           MACRO_19, 
        
           MACRO_20, 
        
           MACRO_21, 
        
           MACRO_22, 
        
           MACRO_23, 
        
           MACRO_24, 
        
           MACRO_25, 
        
           MACRO_26, 
        
           MACRO_27, 
        
           MACRO_28, 
        
           MACRO_29, 
        
           MACRO_30, 
        
           MACRO_31,

wilba · 2022-10-07T11:33:15Z

VIA developers are working on a way to version keycodes etc. so that VIA can be backward compatible with all existing firmware (i.e. their use of the existing keycode values) and accommodate changes to the keycode values. Refactoring how VIA works with respect to keycodes is not trivial and not something that can be done without careful consideration and development time. The decision was made to not include those changes in this PR, causing it to be delayed even more than it has been.

There are plenty of things that can be refactored, improved, etc. and the VIA developers are committed to working on these things. They've put an incredible amount of effort into VIA, including the transition to a web app, so I hope that people can understand the decision to not allow feature creep to delay this PR, which is specifically to do with a change in VIA's protocol and the ability to define custom UI in VIA to support the thing most asked for by users (which is custom lighting).

Refactoring keycodes is definitely top of the list of things in the next breaking change PR, and everyone's suggestions are welcome.

tzarc · 2022-10-08T04:24:24Z

VIA developers are working on a way to version keycodes etc. so that VIA can be backward compatible with all existing firmware (i.e. their use of the existing keycode values) and accommodate changes to the keycode values. Refactoring how VIA works with respect to keycodes is not trivial and not something that can be done without careful consideration and development time. The decision was made to not include those changes in this PR, causing it to be delayed even more than it has been.

Good to hear. QMK has also been working in this area, so in order to ensure you guys have alignment with our direction, we've fast-tracked #18643 to give you ample time to work it in with your plans. I'll ask for a VIA version bump in that PR so that you've got something to check from a compatibility perspective.

quantum/via.h

quantum/via.c

keyboards/geekboards/macropad_v2/keymaps/via/config.h

wilba · 2022-10-31T23:52:24Z

Is there anything else required to get this merged this breaking change round?

tzarc · 2022-11-01T02:19:42Z

Is there anything else required to get this merged this breaking change round?

Not really, it'll likely get included once reviews take place. I'll label it accordingly.
As previously mentioned, #18643 is going to go in this cycle too, so it might be wise sticking a version check on the webapp so as to prevent using older-value keycodes with newer versioned firmware.

I'll try and sort out a review later today.

zvecr · 2022-11-01T02:23:17Z

Is there anything else required to get this merged this breaking change round?

Its still planned for this cycle, with the aim to be merged before the Nov 12 deadline outlined in the docs.

As for what can be done to help it along, all non via related changes to user code should be reverted.

keyboards/work_louder/micro/info.json

keyboards/work_louder/work_board/rules.mk

wilba · 2022-11-02T02:12:29Z

As for what can be done to help it along, all non via related changes to user code should be reverted.

The user code changes are to ensure the keyboard firmware with VIA enabled will build and not be too large. Sometimes this involved fixing little things unrelated to the changes in Core (i.e. it is already broken) but I thought that was worth doing anyway, since those changes will most likely be redundant and merge cleanly if done in another commit.

tzarc

Also pinging @yoichiro -- new VIA version, new macro format/capabilities.
As per the other discussion over on #18643, should probably sort out version checks on both VIA and Remap.

keyboards/cannonkeys/satisfaction75/keymaps/boy_314/keymap.c

keyboards/cannonkeys/satisfaction75/satisfaction75.h

keyboards/custommk/evo70/keymaps/via/config.h

keyboards/custommk/evo70/keymaps/via/keymap.c

keyboards/geekboards/macropad_v2/keymaps/via/config.h

users/talljoe/talljoe.h

users/talljoe/talljoe_keymap.c

quantum/via.c

wilba · 2022-11-08T04:59:26Z

VIA has been updated to use the new int values from #18643, still to do is the refactoring to the changed keycode names and adding new keycodes to the UI.

#18643 changed the VIA protocol version to 11, which was merged to develop. Because of this, for firmware built from develop right now, VIA will now load the V3 definitions and use the changed command handling that is in this PR. Thus currently, things are a bit broken until this PR gets merged.

tzarc · 2022-11-08T05:25:38Z

VIA has been updated to use the new int values from #18643, still to do is the refactoring to the changed keycode names and adding new keycodes to the UI.

#18643 changed the VIA protocol version to 11, which was merged to develop. Because of this, for firmware built from develop right now, VIA will now load the V3 definitions and use the changed command handling that is in this PR. Thus currently, things are a bit broken until this PR gets merged.

I'd recommend making it dynamic if possible -- until the next develop->master merge we might find that some need reordering. Only once it hits master you should consider it to be properly "locked in".

There's some API work coming that will officially set up the publishing pipeline, so you should be able to pull from that URL as required.

github-actions bot added core dependencies keyboard keymap via Adds via keymap and/or updates keyboard for via support labels Aug 30, 2022

drashna changed the base branch from master to develop August 30, 2022 19:13

github-actions bot removed the dependencies label Aug 30, 2022

drashna requested a review from a team August 30, 2022 19:40

Xelus22 mentioned this pull request Sep 18, 2022

Create pi40.json the-via/keyboards#1397

Merged

6 tasks

drashna reviewed Oct 8, 2022

View reviewed changes

quantum/via.h Outdated Show resolved Hide resolved

drashna reviewed Oct 8, 2022

View reviewed changes

quantum/via.h Outdated Show resolved Hide resolved

drashna requested changes Oct 8, 2022

View reviewed changes

drashna requested a review from a team October 8, 2022 16:33

wilba added 5 commits October 10, 2022 13:46

VIA V3

054de5b

remove lib/ugfx

65a6419

Formatting changes

a63c932

Fix qmk lint

35fc1f1

Refactoring the command handlers, formatting

ac6417a

wilba force-pushed the via_v3_pr branch from f63b0b2 to ac6417a Compare October 10, 2022 02:53

fauxpark reviewed Oct 10, 2022

View reviewed changes

keyboards/geekboards/macropad_v2/keymaps/via/config.h Outdated Show resolved Hide resolved

Reduce the muddiness factor of the diffs

87b79b9

github-actions bot added the dependencies label Oct 10, 2022

More reduction of muddiness

be7aaf0

drashna requested a review from a team October 25, 2022 05:43

wilba added 8 commits October 28, 2022 16:45

Merge branch 'develop' into via_v3_pr

ed847e4

Fix build error

2965ffa

Fix firmware size

d87366a

Fix firmware size

d71959d

Fix firmware size

f39fe55

Fix build error

0894924

Merge branch 'develop' into via_v3_pr

0f4b77a

Fix build error

8105f96

tzarc added the breaking_change_2022q4 label Nov 1, 2022

drashna reviewed Nov 1, 2022

View reviewed changes

keyboards/work_louder/micro/info.json Outdated Show resolved Hide resolved

keyboards/work_louder/work_board/rules.mk Outdated Show resolved Hide resolved

Revert changes to Work Louder Micro and Work Board

dcb4c78

Merge branch 'develop' into via_v3_pr

fb17546

tzarc requested changes Nov 5, 2022

View reviewed changes

wilba added 3 commits November 8, 2022 11:59

Rollback the build error fixes

0ea8612

Review changes

94b7efb

Merge branch 'develop' into via_v3_pr

c8ca3ce

tzarc approved these changes Nov 9, 2022

View reviewed changes

tzarc merged commit bc6f8dc into qmk:develop Nov 9, 2022

ramonimbao pushed a commit to ramonimbao/qmk_firmware that referenced this pull request Nov 28, 2022

VIA V3 - The Custom UI Update (qmk#18222)

1dcd04e

zhyu mentioned this pull request Dec 3, 2022

Trackball not working with qmk>= 0.18.0 Yowkees/keyball#162

Closed

elpekenin pushed a commit to elpekenin/qmk_firmware that referenced this pull request Dec 7, 2022

VIA V3 - The Custom UI Update (qmk#18222)

63282de

adophoxia mentioned this pull request Dec 10, 2022

Keychron K6 Pro not detected the-via/releases#211

Open

sigprof mentioned this pull request Apr 22, 2023

RGB Matrix Lighting implementation for Novelkeys NK87 #19422

Closed

14 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

VIA V3 - The Custom UI Update #18222

VIA V3 - The Custom UI Update #18222

wilba commented Aug 30, 2022 •

edited

Loading

drashna commented Aug 30, 2022 •

edited

Loading

adophoxia commented Oct 7, 2022

drashna commented Oct 7, 2022

adophoxia commented Oct 7, 2022

drashna commented Oct 7, 2022

wilba commented Oct 7, 2022

tzarc commented Oct 8, 2022

wilba commented Oct 31, 2022

tzarc commented Nov 1, 2022

zvecr commented Nov 1, 2022

wilba commented Nov 2, 2022

tzarc left a comment

wilba commented Nov 8, 2022

tzarc commented Nov 8, 2022

VIA V3 - The Custom UI Update #18222

VIA V3 - The Custom UI Update #18222

Conversation

wilba commented Aug 30, 2022 • edited Loading

VIA V3 - The Custom UI Update

Types of Changes

Issues Fixed or Closed by This PR

Checklist

drashna commented Aug 30, 2022 • edited Loading

adophoxia commented Oct 7, 2022

drashna commented Oct 7, 2022

adophoxia commented Oct 7, 2022

drashna commented Oct 7, 2022

wilba commented Oct 7, 2022

tzarc commented Oct 8, 2022

wilba commented Oct 31, 2022

tzarc commented Nov 1, 2022

zvecr commented Nov 1, 2022

wilba commented Nov 2, 2022

tzarc left a comment

Choose a reason for hiding this comment

wilba commented Nov 8, 2022

tzarc commented Nov 8, 2022

wilba commented Aug 30, 2022 •

edited

Loading

drashna commented Aug 30, 2022 •

edited

Loading